Methods and systems for picture up-sampling

ABSTRACT

Embodiments of the present invention comprise systems and methods for picture up-sampling using direct interpolation. Some embodiments of the present invention provide an up-sampling procedure designed for the Scalable Video Coding extension of H.264/MPEG-4 AVC.

RELATED REFERENCES

This application claims the benefit of U.S. Provisional PatentApplication No. 60/663,161, entitled “Extended spatial scalability withpicture-level adaptation,” filed Mar. 18, 2005, invented by Shijun Sun;this application also claims the benefit of U.S. Provisional PatentApplication No. 60/683,060, entitled “Direct interpolation forup-sampling in extended spatial scalability,” filed May 20, 2005,invented by Shijun Sun; this application also claims the benefit of U.S.Provisional Patent Application No. 60/686,676, entitled “Deblockingfilter method with reduced complexity for spatial scalable videocoding,” filed Jun. 1, 2005, invented by Shijun Sun.

FIELD OF THE INVENTION

Embodiments of the present invention comprise methods and systems forpicture up-sampling. Some embodiments of the present invention comprisemethods and systems for picture up-sampling for spatially scalable videocoding.

BACKGROUND

H.264/MPEG-4 AVC [Joint Video Team of ITU-T VCEG and ISO/IEC MPEG,“Advanced Video Coding (AVC)—4^(th) Edition,” ITU-T Rec. H.264 andISO/IEC 14496-10 (MPEG4-Part 10), January 2005], which is incorporatedby reference herein, is a video codec specification that uses macroblockprediction followed by residual coding to reduce temporal and spatialredundancy in a video sequence for compression efficiency. Spatialscalability refers to a functionality in which parts of a bitstream maybe removed while maintaining rate-distortion performance at anysupported spatial resolution. Single-layer H.264/MPEG-4 AVC does notsupport spatial scalability. Spatial scalability is supported by theScalable Video Coding (SVC) extension of H.264/MPEG-4 AVC.

The SVC extension of H.264/MPEG-4 AVC [Working Document 1.0 (WD-1.0)(MPEG Doc. N6901) for the Joint Scalable Video Model (JSVM)], which isincorporated by reference herein, is a layered video codec in which theredundancy between spatial layers is exploited by inter-layer predictionmechanisms. Three inter-layer prediction techniques are included intothe design of the SVC extension of H.264/MPEG-4 AVC: inter-layer motionprediction, inter-layer residual prediction, and inter-layer intratexture prediction.

Previously, only dyadic spatial scalability was addressed by SVC. Dyadicspatial scalability refers to configurations in which the ratio ofpicture dimensions between two successive spatial layers is a power of2. New tools that manage configurations in which the ratio of picturedimensions between successive spatial layers is not a power of 2 and inwhich the pictures of the higher level can contain regions that are notpresent in corresponding pictures of the lower level, referred to asnon-dyadic scaling with cropping window, have been proposed.

All of the inter-layer prediction methods comprise picture up-sampling.Picture up-sampling is the process of generating a higher resolutionimage from a lower resolution image. Some picture up-sampling processescomprise sample interpolation. The prior up-sampling process used in theSVC design was based on the quarter luma sample interpolation procedurespecified in H.264 for inter prediction. When applied to spatiallyscalable coding, the prior method has the following two drawbacks: theinterpolation resolution is limited to quarter samples, and thus, is notsupportive of non-dyadic scaling; and half-sample interpolation isrequired in order to get a quarter-sample position making this methodcomputationally cumbersome. A picture up-sampling process that overcomesthese limitations is desired.

SUMMARY

Embodiments of the present invention comprise picture up-samplingaccomplished through direct interpolation using filter coefficientsselected based on the phase of the location of the pixel to beinterpolated.

The foregoing and other objectives, features, and advantages of theinvention will be more readily understood upon consideration of thefollowing detailed description of the invention taken in conjunctionwith the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL DRAWINGS

FIG. 1 is diagram illustrating the geometrical relationship between anenhancement layer and a base layer;

FIG. 2 is a diagram showing the relative location of a sample in anenhancement layer and a base layer;

FIG. 3 is a flow diagram of an embodiment of the present inventioncomprising interpolation filtering in two directions;

FIG. 4 is a diagram illustrating the relationship between macroblocks inan enhancement layer and a base layer; and

FIG. 5 is a diagram illustrating the relationship between macroblocks inan enhancement layer and a base layer.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Embodiments of the present invention will be best understood byreference to the drawings, wherein like parts are designated by likenumerals throughout. The figures listed above are expressly incorporatedas part of this detailed description.

It will be readily understood that the components of the presentinvention, as generally described and illustrated in the figures herein,could be arranged and designed in a wide variety of differentconfigurations. Thus, the following more detailed description of theembodiments of the methods and systems of the present invention is notintended to limit the scope of the invention but it is merelyrepresentative of the presently preferred embodiments of the invention.

Elements of embodiments of the present invention may be embodied inhardware, firmware and/or software. While exemplary embodiments revealedherein may only describe one of these forms, it is to be understood thatone skilled in the art would be able to effectuate these elements in anyof these forms while resting within the scope of the present invention.

For the purposes of this specification and claims, the term “picture”may comprise an array of pixels, a digital image, a subdivision of adigital image, a data channel of a digital image or anotherrepresentation of image data. FIG. 1 shows two pictures corresponding toan image picture: a lower spatial picture 10, also referred to as a basespatial picture or base-layer picture, and a higher spatial picture 100,also referred to as an enhancement spatial picture or enhancement-layerpicture. The base spatial picture 10 may have lower spatial resolutionthan the enhancement spatial picture 100, as shown in FIG. 1. The basespatial picture 10 may not include the same spatial region as that ofthe enhancement spatial picture 100, as shown in FIG. 1. Shown in FIG. 1is a base spatial picture 10 corresponding to a spatial region 110cropped from the enhancement spatial picture 100.

In some embodiments of the present invention, the base spatial pictureand the enhancement spatial picture may correspond to two spatial layersin a scalable video coder/decoder (codec).

The width 101 of the enhancement spatial picture 100 and the height 102of the enhancement spatial picture 100 can be denoted w_(e) and h_(e),respectively. The width 11 and the height 12 of the base spatial picture10 can be denoted w_(b) and h_(b), respectively. The base spatialpicture 10 may be a sub-sampled version of a sub-region 110 of theenhancement spatial picture 100 positioned at enhancement spatialpicture coordinates (x_(orig),y_(orig)) 103. The position 103 representsthe position of the upper-left corner of the cropping window 110. Thewidth 111 and the height 112 of the sub-region 110 can be denoted w_(ex)and h_(ex), respectively. The parameters(x_(orig),y_(orig),w_(ex),h_(ex),w_(b),h_(b)) define the relationshipbetween the higher spatial picture 100 and the lower spatial picture 10.

Picture up-sampling may refer to the generation of a higher spatialresolution image from a lower spatial resolution image. In someembodiments, up-sampling may refer to increasing resolution in anydimension not limited to spatial dimensions or a temporal dimension.FIG. 2 shows a pixel location 220 in a higher spatial resolution image200. In FIG. 2, pixel location 220 has a corresponding location 22 inthe lower spatial resolution image 20. The location 220 may aligndirectly with a pixel location in the lower spatial resolution image 20,or it may not align directly with a pixel location in the lower spatialresolution image 20. In FIG. 2, the location 22 is shown located betweenfour base-layer pixels, 21, 23, 24, and 25.

Some embodiments of the present invention comprise methods and systemsfor direct interpolation of the pixels of the enhancement spatialpicture 200 given the base spatial picture 20 wherein the ratios ofdimensions are not limited to a power of 2. Some embodiments of thepresent invention comprise up-sampling on the entire picture of the basespatial picture 20. Other embodiments of the present invention compriseblock-by-block up-sampling of the base spatial picture 20. Someembodiments of the present invention comprise up-sampling in onedirection followed by up-sampling in another direction.

For a sample position (x,y) in the enhancement spatial picture in unitsof integer samples, the corresponding position (p_(x,L)(X),p_(y,L)(y))in the base spatial picture, in units of $\frac{1}{R_{L}}$samples, may be given by: $\left\{ \begin{matrix}{{p_{x,L}(x)} = {\left\lbrack {{\left( {x - x_{orig}} \right) \cdot w_{b} \cdot R_{L}} + {\frac{R_{L}}{2}\left( {w_{b} - w_{ex}} \right)}} \right\rbrack/w_{ex}}} \\{{p_{y,L}(y)} = {\left\lbrack {{\left( {y - y_{orig}} \right) \cdot h_{b} \cdot R_{L}} + {\frac{R_{L}}{2}\left( {h_{b} - h_{ex}} \right)}} \right\rbrack/h_{ex}}}\end{matrix}\quad \right.$

where the parameters (x_(orig),y_(orig),w_(ex),h_(ex),w_(b),h_(b))define the relationship between the higher spatial picture 100 and thelower spatial picture 10, as in FIG. 1, and R_(L) is the interpolationresolution. Some embodiments comprise one-sixteenth-sample resolutioninterpolation, and in such embodiments R_(L) is 16.

In some embodiments, the corresponding position (p_(x,L)(x),p_(y,L)(y))in the base spatial picture 10 may be given by: $\left\{ \begin{matrix}{{p_{x,L}(x)} = {\left\lbrack {{\left( {x - x_{orig}} \right) \cdot w_{b} \cdot R_{L}} + {\frac{R_{L}}{2}\left( {w_{b} - w_{ex}} \right)}} \right\rbrack//w_{ex}}} \\{{p_{y,L}(y)} = {\left\lbrack {{\left( {y - y_{orig}} \right) \cdot h_{b} \cdot R_{L}} + {\frac{R_{L}}{2}\left( {h_{b} - h_{ex}} \right)}} \right\rbrack//h_{ex}}}\end{matrix}\quad \right.$

where, as above, the parameters(x_(orig),y_(orig),w_(ex),h_(ex),w_(b),h_(b)) define the relationshipbetween the higher spatial picture 100 and the lower spatial picture 10,as in FIG. 1, R_(L) is the interpolation resolution, and “//” comprisesa computationally simplified division operation. The sample positionsmay not be limited to powers of 2, and direct calculation of a sampleposition allows for direct interpolation of picture values at thatsample position.

In some embodiments, interpolating the enhancement-layer image value atsample position (x, y) in the enhancement spatial picture comprises afiltering process. The filtering process may further comprisedetermining interpolation-filter coefficients from a look-up-tablewherein the index into the look-up-table may be related to theinterpolation position determined by (p_(x,L)(x),p_(y,L)(y))

In some embodiments, the interpolation filter may be a 4-tap filter. Insome embodiments, the interpolation filter may be a 6-tap filter. Insome embodiments, the filter coefficients may be derived from thetwo-lobed or three-lobed Lanczos-windowed sinc functions.

Table 1 and Table 2 comprise exemplary look-up-tables ofinterpolation-filter coefficients for a 16-phase 6-tap interpolationfilter wherein the phase corresponds to the interpolation positiondetermined by (p_(x,L)(x),p_(y,L)(y)). TABLE 1 (6-tap) interpolationfilter coefficients phase e[−2] e[−1] e[0] e[1] e[2] e[3] 0 0 0 32 0 0 01 0 −2 32 2 0 0 2 1 −3 31 4 −1 0 3 1 −4 30 7 −2 0 4 1 −4 28 9 −2 0 5 1−5 27 11 −3 1 6 1 −5 25 14 −3 0 7 1 −5 22 17 −4 1 8 1 −5 20 20 −5 1 9 1−4 17 22 −5 1 10 0 −3 14 25 −5 1 11 1 −3 11 27 −5 1 12 0 −2 9 28 −4 1 130 −2 7 30 −4 1 14 0 −1 4 31 −3 1 15 0 0 2 32 −2 0

TABLE 2 (6-tap) interpolation filter coefficients phase e[−2] e[−1] e[0]e[1] e[2] e[3] 0 0 0 32 0 0 0 1 0 −2 32 2 0 0 2 1 −3 31 4 −1 0 3 1 −4 306 −1 0 4 1 −4 28 9 −2 0 5 1 −4 27 11 −3 0 6 1 −5 25 14 −3 0 7 1 −5 22 17−4 1 8 1 −5 20 20 −5 1 9 1 −4 17 22 −5 1 10 0 −3 14 25 −5 1 11 0 −3 1127 −4 1 12 0 −2 9 28 −4 1 13 0 −1 6 30 −4 1 14 0 −1 4 31 −3 1 15 0 0 232 −2 0

Table 3 comprises a look-up-table of interpolation-filter coefficientsfor a 16-phase 4-tap interpolation filter wherein the phase correspondsto the interpolation position determined by (p_(x,L)(x),p_(y,L)(y)).TABLE 3 (4-tap) interpolation filter coefficient phase e[−1] e[0] e[1]e[2] 0 0 128 0 0 1 −4 127 5 0 2 −8 124 13 −1 3 −10 118 21 −1 4 −11 11130 −2 5 −11 103 40 −4 6 −10 93 50 −5 7 −9 82 61 −6 8 −8 72 72 −8 9 −6 6182 −9 10 −5 50 93 −10 11 −4 40 103 −11 12 −2 30 111 −11 13 −1 21 118 −1014 −1 13 124 −8 15 0 5 127 −4

Some embodiments of the current invention are illustrated in FIG. 3.Interpolation in the x- and y-directions may be done in separate passes,300 and 30, respectively. In some embodiments, each pass may beperformed within a macroblock or another sub-division of the image. Inother embodiments, each pass may be performed within the entire image.

For a sample position in the enhancement layer 31, i.e., the location ofan enhancement-layer pixel, the corresponding position in the base layer32 may be determined 301. The offset, or phase, in each direction,y-position phase 33 and x-position phase 34, of the sample in the baselayer from an integer base-layer pixel location may be determined, 302and 303, respectively, from the corresponding base-layer pixel position32 of an enhancement-layer pixel position 31. The offset or phase may bedetermined in units of interpolation resolution. For example, for aninterpolation resolution of one-sixteenth, a phase of 0 may correspondto no offset from a base-layer pixel position. A phase of 8 maycorrespond to an enhancement-layer pixel that falls, in one dimension,half-way between base-layer pixel positions.

The interpolation filter coefficients may be determined by look-up-tablein which the y-position phase 33 may be the index when interpolating inthe y-direction, or the x-position phase 34 may be the index wheninterpolating in the x-direction. The position interpolation center, fora given direction, is the pixel location in the base layer with respectto which the position phase may be measured. In some embodiments of thecurrent invention, the position interpolation center is the pixellocation to which the filter is centered.

FIG. 4 shows the same geometry as that shown in FIG. 1 with an overlayof macroblocks. An exemplary macroblock in the enhancement spatial layer100 may be shown by 140. An exemplary macroblock in the base spatiallayer 10 may be shown by 40. A macroblock 142 in the enhancement spatiallayer 100 may be entirely within the cropped region 110. A macroblock144 in the enhancement spatial layer 100 may be entirely outside thecropped region 110. A macroblock 146 in the enhancement spatial layer100 may be in both the cropped region 110 and outside of the croppedregion 110.

FIG. 5 shows an exemplary overlay of the macroblocks of the up-sampledbase spatial layer 50 on the enhancement spatial layer 100. The dashedlines show the up-sampled base spatial layer macroblocks. Block 51 is anexemplary up-sampled macroblock from the base spatial layer. Somemacroblocks, for example 52, in the enhancement spatial layer 100 haveno base spatial layer corresponding block. Other macroblocks in theenhancement spatial layer 100, have one to several corresponding basespatial layer macroblocks. Block 53 in the enhancement spatial layer 100has one corresponding block, 54, in the base spatial layer. Block 55 inthe enhancement spatial layer 100 has four corresponding blocks, basespatial layer blocks 56, 57, 58, and 59, in the base spatial layer.

An exemplary embodiment of an interpolation process of the presentinvention compatible with the SVC extension of H.264/MPEG-4 AVC isdescribed in the following steps:

Define position (xP,yP) for the upper-left luma sample of a macroblockin the enhancement picture. When chroma_format_idc is not equal to 0,i.e., the chroma channels exist, define position (xC,yC) for theupper-left chroma samples of the same macroblock.

Derive the relative location of the luma macroblock in the lumabase-layer picture, (p_(xL),p_(y,L)), using: $\left\{ {\begin{matrix}{{{x\quad B} = {p_{x,L}\left( {x\quad P} \right)}}\operatorname{>>}4} \\{{{y\quad B} = {p_{y,L}\left( {y\quad P} \right)}}\operatorname{>>}4}\end{matrix}\left\{ {\begin{matrix}{{{x\quad B\quad 1} = \left( {{p_{x,L}\left( {{x\quad P} + 15} \right)} + 15} \right)}\operatorname{>>}4} \\{{{y\quad B\quad 1} = \left( {{p_{y,L}\left( {{y\quad P} + 15} \right)} + 15} \right)}\operatorname{>>}4}\end{matrix}{where}\left\{ \begin{matrix}{{p_{x,L}(x)} = {\left\lbrack {{\left( {x - x_{orig}} \right) \cdot w_{b} \cdot R_{L}} + {\frac{R_{L}}{2}\left( {w_{b} - w_{ex}} \right)}} \right\rbrack//w_{ex}}} \\{{p_{y,L}(y)} = {\left\lbrack {{\left( {y - y_{orig}} \right) \cdot h_{b} \cdot R_{L}} + {\frac{R_{L}}{2}\left( {h_{b} - h_{ex}} \right)}} \right\rbrack//h_{ex}}}\end{matrix} \right.} \right.} \right.$as described above with R_(L)=16. When chroma_format_idc is not equal to0, derive the relative location of the chroma macroblock in the chromabase-layer picture, (p_(x,C),p_(yCL)), using: $\left\{ {\begin{matrix}{{{x\quad C\quad B} = {p_{x,C}\left( {x\quad C} \right)}}\operatorname{>>}4} \\{{{y\quad C\quad B} = {p_{y,C}\left( {y\quad C} \right)}}\operatorname{>>}4}\end{matrix}\left\{ \begin{matrix}{{{x\quad C\quad B\quad 1} = \left( {{p_{x,C}\left( {{x\quad C} + {MbWidthC} - 1} \right)} + 15} \right)}\operatorname{>>}4} \\{{{y\quad C\quad B\quad 1} = \left( {{p_{y,C}\left( {{y\quad C} + {MbHeightC} - 1} \right)} + 15} \right)}\operatorname{>>}4}\end{matrix} \right.} \right.$in which MbWidthC and MbHeightC represent the number of chroma samplesper macroblock (MB) in the horizontal and vertical directions,respectively. And given a chroma sample position (x_(c),y_(c)) in theenhancement picture in units of single chroma samples, its correspondingposition in the base picture (p_(x,c),p_(y,c)) in units of one-sixteenthchroma samples of the base picture can be derived as:$\left\{ \begin{matrix}{{p_{x,c}\left( x_{c} \right)} = {\left\lbrack {{\left( {x_{c} - x_{{orig},c}} \right) \cdot w_{b,c} \cdot R_{C}} + {\frac{R_{C}}{4}\left( {2 + p_{{enh},x}} \right)w_{b,c}} - {\frac{R_{C}}{4}\left( {2 + p_{{base},x}} \right)w_{{ex},c}}} \right\rbrack//w_{{ex},c}}} \\{{p_{y,c}\left( y_{c} \right)} = {\left\lbrack {{\left( {y_{c} - y_{{orig},c}} \right) \cdot h_{b,c} \cdot R_{C}} + {\frac{R_{C}}{4}\left( {2 + p_{{enh},y}} \right)h_{b,c}} - {\frac{R_{C}}{4}\left( {2 + p_{{base},y}} \right)h_{{ex},c}}} \right\rbrack//h_{{ex},c}}}\end{matrix}\quad \right.$in which R_(C)=16, (x_(orig,c),y_(orig,c)) represents the position ofthe upper-left corner of the cropping window in the current picture inunits of single chroma samples of current picture, (w_(b,c),h_(b,c)) isthe resolution of the base picture in units of single chroma samples ofthe base picture, (w_(ex,c), h_(ex,c)) is the resolution of the croppingwindow in units of the single chroma samples of current picture,(p_(base,x),p_(base,y)) represents the relative chroma phase shift ofthe base picture in units of quarter chroma samples of the base picture,and (p_(enh,x),p_(enh,y)) represents the relative chroma phase shift ofthe current picture in units of quarter chroma samples of the currentpicture.Texture Interpolation Process: Prediction

Inputs to this process include:

-   integer luma sample positions in base picture (xB, yB) and (xB1,    yB1),-   a luma sample array for the base picture base_(L)[x, y] with x=−2+xB    . . . (xB1+2) and y=−2+yB . . . (yB1+2),-   when chroma_format_idc is not equal to 0,    -   integer chroma sample positions in base picture (xCB, yCB) and        (xCB1, yCB1),    -   two chroma sample arrays for the base picture base_(Cb)[x, y]        and base_(Cr)[x, y] with x=−2+xCB . . . (xCB1+2) and y=−2+yCB .        . . (yCB1+2).

Outputs of this process include:

-   a luma sample macroblock array pred_(L)[x, y] with x=0 . . . 15 and    y=0 . . . 15,-   when chroma_format_idc is not equal to 0, two chroma sample    macroblock arrays pred_(Cb)[x, y] and pred_(Cr)[x, y] with x=0 . . .    MbWidthC−1 and y=0 . . . MbHeightC−1.

The luma samples pred_(L)[x, y] with x=0 . . . 15 and y=0 . . . 15 arederived as follows.

-   Let temp_(L)[x, y] with x=−2+xB . . . (xB1+2) and y=0 . . . 15 be a    temporary luma sample array.-   Each temp_(L)[x, y] with x=−2+xB . . . (xB1+2) and y=0 . . . 15 is    derived as follows:    -   The corresponding fractional-sample position yf in base layer is        derived as follows:        -   yf=p_(y,L)(y+yP).    -   Let yInt and yFrac be defined as follows:        -   yInt=(yf>>4) and        -   yFrac=yf % 16.    -   Select a six-tap filter e[j] with j=−2 . . . 3 from Table 1        using yFrac as phase, and derive temp_(L)[x, y] as:    -   temp_(L)[x, y]=        -   base_(L)[x, yInt−2]*e[−2]+base_(L)[x,            yInt−1]*e[−1]+base_(L)[x, yInt]*e[0]+base_(L)[x,            yInt+1]*e[1]+base_(L)[x, yInt+2]*e[2]+base_(L)[x,            yInt+3]*e[3].-   Each sample pred_(L)[x, y] with x=0 . . . 15 and y=0 . . . 15 is    derived as follows:    -   The corresponding fractional-sample position xf in base layer is        derived as follows:        -   xf=p_(x,L)(x+xP).    -   Let xInt and xFrac be defined as follows:        -   xInt=(xf>>4) and        -   xFrac=xf % 16.    -   Select a six-tap filter e[j] with j=−2 . . . 3 from Table 1        using xFrac as phase, and derive pred_(L)[x, y] as:    -   pred_(L)[x, y]=Clip1 _(Y)((        -   temp_(L)[xInt−2, y]*e[−2]+temp_(L)[xInt−1,            y]*e[1]+temp_(L)[xInt, y]*e[0]+temp_(L)[xInt+1,            y]*e[1]+temp_(L)[xInt+2, y]*e[2]+temp_(L)[xInt+3,            y]*e[3]+512)/1024),    -   with    -   Clip1 _(Y)(x)=min(max (0, x), (1<<BitDepth_(Y))−1)    -   in which BitDepth_(Y) represents the bit depth of the luma        channel data.

When chroma_format_idc is not equal to 0, the chroma samples predc[x, y](with C denoting each Cb and Cr) with x=0 . . . MbWidthC−1, y=0 . . .MbHeightC−1 are derived as follows.

-   Let tmp1 _(Cb)[x, y] and tmp1 _(Cr)[x, y] with x=−2+xCB . . .    (xCB1+2) and y=0 . . . MbHeightC−1 be temporary chroma sample    arrays.-   Each temp_(C)[x, y], with C as Cb and Cr, x=−2+xCB . . . (xCB1+2),    and y=0 . . . MbHeightC−1 is derived as follows:    -   The corresponding fractional-sample position yfC in base layer        is derived as follows:        -   yfC=p_(y,C)(y+yC).    -   Let yIntC and yFracC be defined as follows:        -   yIntC=(yfC>>4) and        -   yFracC=yfC % 16.    -   Select a six-tap filter e[j] with j=−2 . . . 3 from Table 1        using yFracC as phase, and derive temp_(C)[x, y] as:    -   temp_(C)[x, y]=        -   base_(C)[x, yIntC−2]*e[−2]+base_(C)[x,            yIntC−1]*e[−1]+base_(C)[x, yIntC]*e[0]+base_(C)[x,            yInt_(C)+1]*e[1]+base_(C)[x, yIntC+2]*e[2]+base_(C)[x,            yIntC+3]*e[3].-   Each sample pred_(C)[x, y] with C as Cb and Cr, x=0 . . . MbWidthC−1    and y=0 . . . MbHeightC−1 is derived as follows:    -   The corresponding fractional-sample position xfC in base layer        is derived as follows:        -   xfC=p_(x,C)(x+xC).    -   Let xIntC and xFracC be defined as follows        -   xIntC=(xfC>>4) and        -   xFracC=xfC % 16.    -   Select a six-tap filter e[j] with j=−2 . . . 3 from Table 1        using xFracC as phase, and derive predc[x, y] as        -   pred_(C)[x, y]=Clip1 _(C)((        -   temp_(C)[xIntC−2, y]*e[−2]+temp_(C)[xInt_(C)−1,            y]*e[−1]+temp_(C)[xInt_(C), y]*e[0]+temp_(C)[xInt_(C)+1,            y]*e[1]+temp_(C)[xInt_(C)+2, y]*e[2]+temp_(C)[xInt_(C)+3,            y]*e[3]+512)/1024),    -   with    -   Clip1 _(C)(x)=min(max (0, x), (1<<BitDepth_(C))−1)    -   in which BitDepth_(C) represents the bit depth of the chroma        channel data.        Texture Interpolation Process: Residual

Inputs to this process include:

-   integer luma sample positions in base picture, basePic, (xB,yB) and    (xB1, yB1),-   a luma residual sample array resBase_(L)[x, y] with x=−xB . . . xB1    and y=yB . . . yB1,-   when chroma_format_idc is not equal to 0,    -   integer chroma sample positions in base picture, basepic, (xCB,        yCB) and (xCB1, yCB1),    -   two chroma residual sample arrays resBase_(Cb)[X, y] and        resBase_(Cr)[x, y] with x=xCB . . . xCB1 and y=yCB . . . yCB1.

Outputs of this process include:

-   a luma sample array resPred_(L)[X, y] with x=0 . . . 15 and y=0 . .    . 15.-   when chroma_format_idc is not equal to 0, two chroma sample arrays    resPred_(Cb)[x, y] and resPred_(Cr)[x, y] with x=0 . . . MbWidthC−1    and y=0 . . . MbHeightC−1.

The luma residual samples resPred_(L)[x, y] with x=0 . . . 15 and y=0 .. . 15 are derived as follows.

-   Let temp_(L)[X, y] with x=xB . . . xB1 and y=0 . . . 15 be a    temporary luma sample array.-   Each temp_(L)[x, y] with x=xB . . . xB1 and y=0 . . . 15 is derived    as follows:    -   The corresponding fractional-sample position yf in base layer is        derived as follows:        -   yf=p_(y,L)(y+yP).    -   Let yInt and yFrac be defined as follows:        -   yInt=(yf>>4) and        -   yFrac=yf % 16.    -   Derive temp_(L)[x, y] as        -   temp_(L)[x, y]=resBase_(L)[x, yInt]*(16−yFrac)+resBaseL[x,            yInt+1]*yFrac.-   Each residual sample resPredL[x, y] with x=0 . . . 15 and y=0 . . .    15 is derived as follows:    -   The corresponding fractional-sample position xf in base layer is        derived as follows:        -   xf=p_(x,L)(x+xP).    -   Let xInt and xFrac be defined as follows:        -   xInt=(xf>>4) and        -   xFrac=xf % 16.    -   Derive resPred_(L)[x, y] as        -   resPred_(L)[x, y]=            -   (temp_(L)[xInt, y]*(16−xFrac)+temp_(L)[xInt+1,                y]*xFrac)/256.

When chroma_format_idc is not equal to 0, the chroma residual samplesresPredc[x, y] (with C being Cb or Cr) with x=0 . . . MbWidthC−1, y=0 .. . MbHeightC−1 are derived as follows.

-   Let tmp1 _(Cb)[x, y] and tmp1 _(Cr)[x, y] . . . with x=xCB . . .    xCB1 and y=0 . . . MbHeightC−1 be temporary chroma sample arrays.-   Each temp_(C)[x, y] with C as Cb and Cr, x=xCB . . . xCB1, and y=0 .    . . MbHeightC−1 is derived as follows:    -   The corresponding fractional-sample position yfC in base layer        is derived as follows:        -   yfC=p_(y,C)(y+yC).    -   Let yIntC and yFracC be defined as follows:        -   yIntC=(yfC>>4) and        -   yFracC=yfC % 16.    -   Derive temp_(C)[x, y] as        -   temp_(C)[x, y]=resBase_(C)[x, yIntC]*(16−yFracC)+resbasec[x,            yintC+1]*yFracC.-   Each sample resPre_(C)[x, y] with C as Cb and Cr, x=0 . . .    MbWidthC−1 and y=0 . . . MbHeight−1 is derived as follows:    -   The corresponding fractional-sample position xfC in base layer        is derived as follows:        -   xfC=p_(x,C)(x+xC).    -   Let xIntC and xFracC be defined as follows:        -   xIntC=(xfC>>4) and        -   xFracC=xfC % 16.    -   Derive resPred_(C)[x, y] as        -   resPred_(C)[x, y]=        -   (temp_(C)[xIntC, y]*(16−xFracC)+temp_(C)[xIntC+1,            y]*xFracC)/256.

The terms and expressions which have been employed in the foregoingspecification are used therein as terms of description and not oflimitation, and there is no intention in the use of such terms andexpressions of excluding equivalence of the features shown and describedor portions thereof, it being recognized that the scope of the inventionis defined and limited only by the claims which follow.

1. A method for picture up-sampling from a base-layer picture comprisingbase-layer image samples, base_(L), to an enhancement-layer picturecomprising enhancement-layer image samples, pred_(L), comprising: a)defining a position, (xP,yP), for an upper-left sample of a block ofsaid enhancement-layer picture; b) deriving a relative location,xB,yB,xB1,yB1, of said block of said enhancement-layer picture in saidbase-layer picture; c) producing a sample block array, pred_(L)[x, y],wherein said producing comprises: i) forming a temporary sample array,temp_(L)[x, y]; ii) deriving an entry in said temporary sample arraywherein said deriving comprises: (1) calculating a correspondingfractional-sample y-position, yf, in said base-layer picture; (2)calculating a y-position interpolation center, yInt, based on yf; (3)calculating a y-position phase based on yf; (4) obtaining a first set offilter coefficients for a first six-tap filter wherein said obtaining ofsaid first set of filter coefficients is based on said y-position phase;and (5) deriving said entry temp_(L)[x, y] according to: temp_(L)[x, y]=base_(L)[x, yInt−2]*e[−2]+base_(L)[x, yInt−1]*e[−1]+base_(L)[x,yInt]*e[0]+base_(L)[x, yInt+1]*e[1]+base_(L)[x, yInt+2]*e[2]+base_(L)[x,yInt+3]*e[3], wherein e[−2], e[−1], e[0], e[1], e[2], and e[3] representsaid first set of filter coefficients for said first six-tap filter; andiii) deriving an entry in said sample block array, pred_(L)[x, y],wherein said deriving comprises: (1) calculating a correspondingfractional-sample x-position, xf, in said base-layer picture; (2)calculating an x-position interpolation center, xInt, based on xf; (3)forming an x-position phase based on xf; (4) obtaining a second set offilter coefficients for a second six-tap filter wherein said obtainingof said second set of filter coefficients is based on said x-positionphase; and (5) deriving said entry pred_(L)[x, y] according to:pred_(L)[x, y]= Clip1 _(Y)(( temp_(L)[xInt−2, y]*e[−2]+temp_(L)[xInt−1,y]*e[−1]+temp_(L)[xInt, y]*e[0]+temp_(L)[xInt+1,y]*e[1]+temp_(L)[xInt+2, y]*e[2]+temp_(L)[xInt+3, y]*e[3]+512)/1024),wherein e[−2], e[−1], e[0], e[1], e[2], and e[3] represent said secondset of filter coefficients for said second six-tap filter.
 2. The methodof claim 1 further comprising defining an interpolation resolution. 3.The method of claim 2 wherein said interpolation resolution is 1/16(one-sixteenth) of an image sample.
 4. The method of claim 1 whereinsaid base-layer image samples comprise luminance samples.
 5. The methodof claim 1 wherein said base-layer image samples comprise chrominancesamples.
 6. The method of claim 1 wherein said obtaining of said firstset of filter coefficients comprises accessing a look-up-table.
 7. Themethod of claim 1 wherein said obtaining of said second set of filtercoefficients comprises accessing a look-up-table.
 8. The method of claim6 wherein said look-up-table comprises samples of a Lanczos-windowedsinc function.
 9. The method of claim 7 wherein said look-up-tablecomprises samples of a Lanczos-windowed sinc function.
 10. The method ofclaim 1 wherein said block is a macroblock of dimension 16 by
 16. 11.The method of claim 1 wherein said block has the same spatial dimensionsas said enhancement-layer picture.
 12. A method for picture up-samplingfrom a lower resolution picture to a higher resolution picture saidmethod comprising: a) deriving a lower resolution picture locationcorresponding to a sample position in said higher resolution picture; b)calculating a first position interpolation center in a first directionbased on said location; c) calculating a first position phase in saidfirst direction based on said location; d) calculating a second positioninterpolation center in a second direction based on said location; e)calculating a second position phase in said second direction based onsaid location; f) obtaining a first set of filter coefficients whereinsaid obtaining of said first set of filter coefficients is based on saidfirst position phase; g) obtaining a second set of filter coefficientswherein said obtaining of said second set of filter coefficients isbased on said second position phase; h) filtering said lower resolutionpicture in said first direction using a first filter comprising saidfirst set of filter coefficients thereby producing a first filteredpicture; and i) filtering said first filtered picture in said seconddirection using a second filter comprising said second set of filtercoefficients thereby producing said higher resolution picture.
 13. Themethod of claim 12 further comprising defining an interpolationresolution.
 14. The method of claim 13 wherein said interpolationresolution is 1/16 (one-sixteenth) of an image sample.
 15. The method ofclaim 12 wherein said obtaining of said first set of filter coefficientscomprises accessing a look-up-table.
 16. The method of claim 12 whereinsaid obtaining of said second set of filter coefficients comprisesaccessing a look-up-table.
 17. The method of claim 15 wherein saidlook-up-table comprises samples of a Lanczos-windowed sinc function. 18.The method of claim 16 wherein said look-up-table comprises samples of aLanczos-windowed sinc function.
 19. A system for picture up-samplingfrom a lower resolution picture to a higher resolution picture saidsystem comprising: a) a relative-location deriver for deriving a lowerresolution picture location corresponding to a sample position in saidhigher resolution picture; b) a first position-interpolation-centercalculator for calculating a first position-interpolation center in afirst direction based on said location; c) a first position-phasecalculator for calculating a first position phase in said firstdirection based on said location; d) a secondposition-interpolation-center calculator for calculating a secondposition-interpolation center in a second direction based on saidlocation; e) a second position-phase calculator for calculating a secondposition phase in said second direction based on said location; f) afirst filter-coefficient selector for obtaining a first set of filtercoefficients wherein said obtaining of said first set of filtercoefficients is based on said first position phase; g) a secondfilter-coefficient selector for obtaining a second set of filtercoefficients wherein said obtaining of said second set of filtercoefficients is based on said second position phase; h) a first filtercomprising said first set of filter coefficients for filtering saidlower resolution picture in said first direction thereby producing afirst filtered picture; and i) a second filter comprising said secondset of filter coefficients for filtering said first filtered picture insaid second direction thereby producing said higher resolution picture.20. The system of claim 19 further comprising an interpolationresolution.
 21. The system of claim 20 wherein said interpolationresolution is 1/16 (one-sixteenth) of an image sample.
 22. The system ofclaim 19 wherein said obtaining of said first set of filter coefficientscomprises a look-up-table.
 23. The system of claim 19 wherein saidobtaining of said second set of filter coefficients comprises alook-up-table.
 24. The system of claim 22 wherein said look-up-tablecomprises samples of a Lanczos-windowed sinc function.
 25. The system ofclaim 23 wherein said look-up-table comprises samples of aLanczos-windowed sinc function.
 26. The system of claim 19 furthercomprising an H.264/MPEG-4 AVC video codec.